Broadcast selection

ABSTRACT

Disclosed technology includes a sensory prosthesis configured to automatically select a broadcast channel based on a comparison with a signal from a sensor of the sensory prosthesis. In an example, a sound processor automatically connects to an appropriate wireless broadcast audio channel by comparing the sound the sound processor receives on a microphone with the audio the sound processor receives from each of the wireless broadcast channels that the sound processor can receive. If the sound processor finds a match between the sound from the microphone and the sound from a broadcast, then the sound processor automatically selects the matching wireless broadcast channel. The sound processor then provides auditory stimulation to the recipient based on the selected broadcast channel.

BACKGROUND

Medical devices have provided a wide range of therapeutic benefits torecipients over recent decades. Medical devices can include internal orimplantable components/devices, external or wearable components/devices,or combinations thereof (e.g., a device having an external componentcommunicating with an implantable component). Medical devices, such astraditional hearing aids, partially or fully-implantable hearingprostheses (e.g., bone conduction devices, mechanical stimulators,cochlear implants, etc.), pacemakers, defibrillators, functionalelectrical stimulation devices, and other medical devices, have beensuccessful in performing lifesaving and/or lifestyle enhancementfunctions and/or recipient monitoring for a number of years.

The types of medical devices and the ranges of functions performedthereby have increased over the years. For example, many medicaldevices, sometimes referred to as “implantable medical devices,” nowoften include one or more instruments, apparatus, sensors, processors,controllers or other functional mechanical or electrical components thatare permanently or temporarily implanted in a recipient. Thesefunctional devices are typically used to diagnose, prevent, monitor,treat, or manage a disease/injury or symptom thereof, or to investigate,replace or modify the anatomy or a physiological process. Many of thesefunctional devices utilize power and/or data received from externaldevices that are part of, or operate in conjunction with, implantablecomponents.

SUMMARY

In an example, there is a method comprising: determining that abroadcast sample and a sensor sample match; and selecting a broadcastassociated with the broadcast sample responsive to the broadcast sampleand the sensor sample match.

In another example, there is a computer-readable medium havinginstructions stored thereon that, when executed by one or moreprocessors, cause the one or more processors to: check for a set ofcandidate broadcasts; and for each respective broadcast of the set ofcandidate broadcasts: obtain a broadcast sample from the respectivebroadcast; obtain a sensor sample from a sensor; compare the broadcastsample and the sensor sample; and select the respective broadcastresponsive to the comparing indicating that the broadcast sample and thesensor sample match.

In a further example, there is a system comprising: a sensoryprosthesis; a sensor; a receiver; and one or more processors. The one ormore processors are configured to: obtain a broadcast sample from thereceiver; obtain a sensor sample from the sensor; determine whether thebroadcast sample and the sensor sample match; and select, as a sourceused by the sensory prosthesis to cause a person to experience a sensorypercept, a broadcast associated with the broadcast sample responsive tothe broadcast sample and the sensor sample matching.

BRIEF DESCRIPTION OF THE DRAWINGS

The same number represents the same element or same type of element inall drawings.

FIG. 1 illustrates an example system that includes a user device, abroadcast system, and a secondary device.

FIG. 2 illustrates an example method.

FIG. 3 illustrates a computer-readable medium having instructions storedthereon that, when executed by one or more processors, cause the one ormore processors to perform a method that includes one or moreoperations.

FIG. 4 illustrates one or more processors configured to perform one ormore operations.

FIG. 5 illustrates an example of a suitable computing system with whichone or more of the disclosed examples can be implemented.

FIG. 6 is a functional block diagram of an implantable stimulator systemthat can benefit from the technologies described herein.

FIG. 7 illustrates an example cochlear implant system that can benefitfrom use of the technologies disclosed herein.

FIG. 8 illustrates a retinal prosthesis system that comprises anexternal device, a retinal prosthesis and a mobile computing device.

DETAILED DESCRIPTION

Disclosed technology includes a user device configured to automaticallyselect a broadcast channel based on a comparison with a signal from oneor more sensors of the user device, such as a consumer device or asensory prosthesis. Example consumer devices include headphones,earbuds, personal sound amplification products, wireless earbuds, orother consumer devices. Example sensory prostheses include auditoryprostheses and visual prostheses.

A sensory prosthesis provides sensory stimulation to its recipient. Suchstimulation is typically provided based on data obtained from one ormore sensors of the sensory prosthesis. An auditory prosthesis, such asa hearing aid or cochlear implant, receives audio data from one or moremicrophones and uses the audio data to deliver stimulation, forinstance. But some sensory prostheses can be configured to providestimulation based on data received from one or more wirelesstransmissions. For instance, a recipient of a visual prosthesiswirelessly streams video from the recipient's phone, such that thestimulation from the visual prosthesis is based on the wireless streamfrom the phone in addition to or instead of data from a camera of thevisual prosthesis. In many examples, data received wirelessly fromanother device is of higher quality than data produced by theprosthesis's sensors (e.g., because the wireless data remains digitalrather than being converted to an analog signal, transmitted in ananalog manner, received as an analog signal, and then is converted to adigital signal). For instance, a theater may provide audio via standardtheater speakers as well as audio via a wireless broadcast usingBLUETOOTH. A recipient of a cochlear implant watching a movie at thattheater may prefer to receive auditory stimulation based on the wirelessbroadcast compared to audio received through one or more microphones ofthe recipient's sensory prosthesis. In such an instance, the recipientwould typically need to be aware of the broadcast and then manuallyselect to receive the transmission via the broadcast.

A non-limiting example of a device being configured to operate based onoutput from a sensor or output from an accessory device is described inU.S. Pat. No. 8,706,245, which is hereby incorporated herein byreference in its entirety for any and all purposes. The patent describesthat, in a first mode of operation, a hearing prosthesis receives amicrophone input and produces an output based on the microphone input.In a second mode of operation, the hearing prosthesis detects anaccessory input signal and switches to an accessory input mode. Thesecond mode of operation produces an output that is based at least inpart on the accessory input signal (e.g., as transmitted overBLUETOOTH). When the accessory input signal is not detected, the hearingprosthesis operates in microphone operation mode.

In an example of technology disclosed herein, a sound processor of anauditory prosthesis automatically connects to an appropriate wirelessbroadcast audio channel (e.g., a BLUETOOTH audio channel) by comparingthe sound the sound processor receives from a microphone of the soundprocessor with audio that the sound processor receives from each of thewireless broadcast channels that the sound processor can receive. If thesound processor finds a match between the sound from the microphone andthe sound from a broadcast, then the sound processor automaticallyselects the matching wireless broadcast channel. The sound processorthen provides auditory stimulation to the recipient based on broadcastaudio from the selected broadcast channel in addition to or instead ofsensor audio from one or more microphones.

An example system usable with disclosed is shown in FIG. 1 .

Example System

FIG. 1 illustrates an example system 100 that includes a user device102, a broadcast system 150, and a secondary device 190.

The user device 102 is a device of a user that provides stimulation to auser of the user device 102 to cause a sensory percept. For instance,the sensory precepts are related to one or more of the five traditionalsenses (vision, hearing, touch, taste, and smell) and/or one or moreadditional senses (e.g., proprioception). For ease of understanding,many examples herein are discussed in the context of auditory perceptsand visual percepts. The user device 102 can take any of a variety offorms, such as a consumer device or a medical prosthesis. Exampleconsumer devices include headphones, earbuds, personal soundamplification products, wireless earbuds, or other consumer devices.Example prostheses include auditory prostheses and visual prostheses.Example auditory prostheses include one or more prostheses selected fromthe group consisting of: a cochlear implant, an electroacoustic device,a percutaneous bone conduction device, a passive transcutaneous boneconduction device, an active transcutaneous bone conduction device, amiddle ear device, a totally-implantable auditory device, amostly-implantable auditory device, an auditory brainstem implantdevice, a hearing aid, and a tooth-anchored hearing device. Examplevisual prostheses include bionic eyes.

Then user device 102 can include any of a variety of components based onits form and function. In the illustrated example, the user device 102includes one or more processors 110, memory 116, a sensor 120, areceiver 130, and a stimulator 140.

The one or more processors 110 are one or more hardware or softwareprocessing units (e.g., Central Processing Units) that can obtain andexecute instructions, such as to communicate with and control theperformance of other components of the user device 102 or the system100. In addition or instead, the one or more processors 110 can includemicrocontrollers configured to perform one or more operations.

The memory 116 is one or more software- or hardware-basedcomputer-readable storage media operable to store information accessibleby the one or more processors 110. The memory 116 can store, among otherthings, instructions executable by the one or more processors 110 toimplement applications or cause performance of operations describedherein, as well as other data. The memory 116 can be volatile memory(e.g., RAM), non-volatile memory (e.g., ROM), or combinations thereof.The memory 116 can include transitory memory or non-transitory memory.The memory 116 can also include one or more removable or non-removablestorage devices. In examples, the memory 116 includes RAM, ROM, EEPROM(Electronically-Erasable Programmable Read-Only Memory), flash memory,optical disc storage, magnetic storage, solid state storage, or anyother memory media usable to store information for later access. Inexamples, the memory 116 encompasses a modulated data signal (e.g., asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal), such as a carrierwave or other transport mechanism and includes any information deliverymedia. By way of example, and not limitation, the memory 116 includeswired media such as a wired network or direct-wired connection, andwireless media such as acoustic, RF, infrared and other wireless mediaor combinations thereof.

The sensor 120 is a component that generates signals based on sensedoccurrences, such as data regarding the environment around the userdevice 102. In some examples, the sensors 120 are configured to obtaindata for the generation of stimulation via the stimulator 140. Inaddition or instead, the sensor 120 can provide other functionality tothe user device 102, such as providing sensing for a communication(e.g., a call or noise cancelation). In an example, the sensor 120 is amicrophone. In another example, the sensor 120 is a camera.

The receiver 130 is a component configured to usably receive awirelessly transmitted signal. Typically, the wirelessly transmittedsignal is transmitted in a from not readily perceptible by a person(e.g., because the transmitted signal is outside of the range of typicalhuman senses). In some examples, the receiver 130 is or includes atransmitter or transceiver. In some examples, the receiver 130 includesan antenna, such as an antenna configured to receive radio waves ofbetween 76.0 MHz and 108.0 MHz, between 2.40 GHz and 2.50 GHz, orbetween 5.00 GHz and 6.00 GHz. In examples, the receiver 130 isconfigured to receive WI-FI signals, BLUETOOTH signals, or FM signals.In examples, the receiver 130 includes multiple components (e.g., anantenna and a processor). In an example, the receiver 130 is in asystem-on-a-chip configuration.

The stimulator 140 is a component configured to cause the recipient ofthe user device 102 to experience a sensory percept, such as a visualpercept or a hearing percept.

In an example, the stimulator 140 is a component configured to providestimulation to a recipient's auditory system to cause a hearing perceptto be experienced by the recipient. Examples of components usable forauditory stimulation include components for generating air-conductedvibrations (e.g., a speaker), components for generating bone-conductedvibration (e.g., an actuator coupled to a conducting pad or anchor),components for generating electrical stimulation (e.g., one or moreelectrodes), other components, or combinations thereof.

In an example, the stimulator 140 is a component configured to providevisual stimulation to a recipient's visual system to cause a visualpercept to be experienced by the recipient. Examples of componentsusable for visual stimulation include components for generating visiblelight (e.g., a display), components for generating electricalstimulation (e.g., one or more electrodes), other components, orcombinations thereof.

Although the user device 102 is shown as a single structure havingmultiple components, the user device 102 can take other forms. In someexamples, the user device 102 is split into an implantable component(e.g., a cochlear implant implanted in the recipient) and an externalcomponent. In further examples, the user device 102 is split intoseparate left and right components to fit in the recipient's ears (e.g.,in the form of earbuds). The user device 102 can further includecompanion devices, such as a charging case or battery pack. Exampleimplementations of the user device are shown and described in relationto FIGS. 6-8 .

The broadcast system 150 is a device or system of devices that providescontent via a broadcast signal 10 and a signal 20 of another modality(e.g., via air conducted sound or via visual signal). As illustrated,the broadcast system 150 includes a display 160, a speaker 170, and atransmitter 180. The broadcast system 150 can take other forms andconfigurations.

In the illustrated example, the broadcast system 150 is a televisionthat provides a visual signal 20 via the display 160, an air-conductedaudio signal 20 via the speaker 170, and broadcasts an audio signalsimilar to the audio signal 20 via a broadcast signal 10. For instance,the broadcast signal 10 uses BLUETOOTH, WI-FI, FM, or another technologyto encode and broadcast the sound that also makes up the audio signal 20via the transmitter 180. In addition or instead, the broadcast signal 10encodes some or all of the visual signal 20 provided by the display 160.Broadcast device 150 technology is applicable to a variety ofcircumstances, such as at a home, gym, restaurant, theater, or otherlocation. Similar approaches are applicable to use in classrooms orlecture halls where a person acts as the broadcaster with their naturalspeaking voice (amplified or unamplified) corresponds to the audiosignal 20 and a broadcast system (e.g., via an induction loop, FMtransmitter, or BLUETOOTH transmission via isochronous channels)transmits the broadcast signal 10 to people who want to receive thebroadcast signal (e.g., individuals having hearing aids). Furtherexamples include transportation facilities (e.g., airports, trainstations, or bus terminals) or buildings (e.g., campuses or malls)having a public address system that provides audio signals 20 viaspeakers and may also provide inaudible broadcast signals 10. Thebroadcast system 150 can take any of a variety of forms to fit suchcircumstances. In a further example, the broadcast system 150 is abroadcast system for an airport that includes multiple speakers 170 invarious areas of the airport (e.g., for making announcements) and hasmultiple displays 160 in various areas of the airport (e.g., fordisplaying the announcements). The airport can further includetransmitters 180 in various locations to provide the audio or visualcomponent of the announcements to devices that tune into or connect tothe broadcast system 150.

The secondary device 190 is a device other than the user device 102 andthe broadcast system 150. The secondary device 190 can nonetheless alsobe a device of the user or recipient of the user device 102 (e.g., adevice that the user owns, controls, or operates). For example, thesecondary device 190 is a computing device associated with the recipientof the user device 102 such as a phone, tablet, or wearable device,among other forms.

In the illustrated example, the secondary device 190 includes, amongother components, one or more processors 110, memory 116, one or moresensors 120, a receiver 130 and a user device application 152 (e.g.,encoded in the memory 116 and executed by the one or more processors 110of the secondary device 190) memory 116, and a user device application192. The one or more processors 110, sensors 120, and memory 116 can beas described above in relation to the user device 102. The user deviceapplication 192 is a software application that operates on the secondarydevice 190 and cooperates with the user device 102 directly or via anintermediary device. In an example, the user device application 192controls the user device 102 (e.g., based on input received from therecipient) and obtains data from the user device 102 and other devices.The secondary device 190 connects to the user device 102 using, forexample, a wireless radiofrequency communication protocol (e.g.,BLUETOOTH). The secondary device 190 transmits data to or receives datafrom the user device 102 over such a connection. In examples where theuser device 102 is an auditory device, the secondary device 190 canstream audio to user device 102 for stimulating the recipient of theuser device 102 using the stimulator 140. In some examples, the userdevice application 192 provides a user interface over with the user canmodify settings of the user device 102 as well as one or more parametersof techniques described herein.

One or more of the components of the system 100 can cooperate orcoordinate to perform one or more operations described herein, such asthe example method of FIG. 2 .

Example Method

FIG. 2 illustrates an example method 200. The operations of the method200 can be performed by one or more of the components of the system 100.In some examples, the method 200 is performed when stimulation providedby the user device 102 is not being based on a broadcast. In someexamples, the method 200 is performed when the stimulation provided bythe user device 102 is being based on a broadcast, such as to ensurethat the broadcast is still relevant.

Operation 210 includes receiving a broadcast 212. For example, thebroadcast 212 is received by the receiver 130 of the user device 102 orthe secondary device 190. In an example, the broadcast 212 is receivedas a wireless signal, such as radio waves having a frequency between 2.4GHz and 2.5 GHz that encode a signal.

In an example, the broadcast 212 is generated by the transmitter 180 ofthe broadcast system 150. The broadcast 212 is able to be sent using anyof a variety of protocols, such as WI-FI (e.g., broadcast as radio wavesof between 2.40 GHz and 2.50 GHz or between 5.00 GHz and 6 GHz thatencode a signal in compliance with IEEE 802.11), BLUETOOTH (e.g.,broadcast as radio waves of between 2.4 GHz and 2.5 GHz that encode asignal in compliance with the BLUETOOTH 5.2 specification), FM (e.g.,broadcast as radio waves of between 76.0 MHz and 108.0 MHz that encode asignal by modulating frequency), other techniques, or combinationsthereof.

In some examples, receiving the broadcast 212 includes performing one ormore preparatory steps. In an example, prior to receiving the broadcast212, one or more components of the user device 102 tune to a frequencyof the broadcast 212. In an example, a portion of the broadcast 212 isreceived, and then one or more components of the user device 102 preformone or more operations to connect to the broadcast 212 fully, such asoperations that permit the user device 102 to usefully receive theintended content of the broadcast 212 (e.g., the audio or visualcontent). For instance, the broadcast 212 provides an identifier of abroadcast 212 (e.g., a WI-FI service set identifier) and the user device102 takes one or more actions to connect to or pair with the broadcast212 or the source of the broadcast 212. In some instances, the userdevice 102 authenticates with the source of the broadcast 212 such as byproviding an identifier or passcode.

Operation 220 includes obtaining a broadcast sample 222 from thebroadcast 212. In an example, the obtaining includes storing a portionof broadcast content provided over the broadcast 212. For example, thebroadcast sample 222 is a portion of audio or video data provided overthe broadcast 212. The broadcast sample 222 can be any of a variety oflengths.

In many examples, the broadcast 212 transmits a streaming media signalconveying media content configured to be consumed (e.g., played)substantially contemporaneously (e.g., in real time) with receiving thestreaming media signal, rather than, for example, transmitting a mediacontent file for playback at a later time (e.g., providing the mediacontent file for the receiving device to download). Nonetheless, whilein some examples the broadcast sample 222 is a short snippet of thestreaming media signal, in other examples, the broadcast sample 222 canbe obtained as a relatively longer, buffered or downloaded signal.

Operation 230 includes obtaining a broadcast sample fingerprint 232 fromthe broadcast sample 222. The broadcast sample fingerprint 232 is aportion of the broadcast sample 222 or a representation of the broadcastsample 222 configured for comparison with another fingerprint (e.g., asensor sample fingerprint). The broadcast sample fingerprint 232 can beformed in any of a variety of ways.

In an example, obtaining the broadcast sample fingerprint 232 includescharacteristic landmarks or features within the sample. Such landmarkscan be, for example, particularly high or low volume, frequency, orenergy sections within the broadcast sample 222. The broadcast samplefingerprint 232 can further be applied to patterns within the broadcastsample 222. For instance, the fingerprint 232 can indicate that certainlandmarks occur a particular time apart and have certaincharacteristics. In a specific example, the fingerprint 232 is dataindicating that the broadcast sample 222 includes a 0.05 second segmenthaving a volume 50% greater than a mean volume of the sample 222, then a0.1 second segment having a volume 50% less than the mean volume, andthen a 0.25 second segment having a volume 25% greater than the meanvolume of the sample. In an example, the broadcast sample fingerprint232 is a spectrogram or is derived from a spectrogram.

In an example, obtaining the broadcast sample fingerprint 232 includesperforming speech-to-text processing on the broadcast sample. Inaddition or instead, obtaining the broadcast sample fingerprint 232includes performing content recognition on the broadcast sample. Forinstance, the broadcast sample 222 is provided as input to an artificialintelligence system configured to recognize sounds (e.g., identifyingsounds as being of a particular type or category, such as birdsong,human speech, music, etc.) or visuals (e.g., visuals of a particulartype or category, such as birds, people, or musical instruments)conveyed by the broadcast sample 222. The recognized output can be usedas the broadcast sample fingerprint 232.

Other techniques can be used to generate the fingerprint 232, such asvarious techniques for generating fingerprints that are known in theart. In an example, a perceptual hashing algorithm is used. Further,some organizations offer open source libraries for comparing content orgenerating perceptual hashes. Examples include PHASH by AETILIUS, INC.

Operation 240 includes obtaining a sensor sample 242. In an example, theobtaining includes storing a portion of sensor data provided by a sensor120. Where the sensor 120 is a camera, obtaining the sensor sample 242includes obtaining the sensor sample 242 with or from a camera. Wherethe sensor 120 is a microphone, obtaining the sensor sample 242 includesobtaining the sensor sample 242 with or from a microphone. Depending onthe implementation, different devices obtain the sensor sample 242, suchas the user device 102 implemented as a visual prosthesis, consumerdevice, or auditory prosthesis. In another implementation, the secondarydevice 190 obtains the sensor sample 242, such as the secondary device190 implemented as a phone, tablet, laptop, or wearable computingdevice. While in some examples the sensor sample 242 is a short snippetof the streaming media signal, in other examples, the sensor sample 242can be obtained as a relatively longer, buffered or downloaded signal.

In some examples, operation 240 includes one or more aspects describedbelow in conjunction with operation 340 of FIG. 3 .

Operation 250 includes obtaining a sensor sample fingerprint 252 fromthe sensor sample 242. In many examples, the sensor sample fingerprint252 is obtained using the same or substantially similar technique to theone or more techniques used to generate the broadcast sample fingerprint232 as described above in conjunction with operation 230.

Operation 260 includes determining whether the broadcast sample 222 andthe sensor sample 242 match. In an example, the operation 260 determinesthat the broadcast sample 222 and the sensor sample 242 match responsiveto determining that the broadcast sample fingerprint 232 and the sensorsample fingerprint 252 have a level of similarity within a thresholdamount. In some examples, the threshold is configurable by the user(e.g., via a setting screen in the user device application 192). In someexamples, the threshold is configured to overcome a level ofenvironmental noise around the user. For instance, the sensor sample 242is more likely to pick up ambient noise that would not be present in thebroadcast sample 222. Thus, the comparison can be configured to accountfor such a difference. In some examples, the threshold is automaticallyadjusted based on an amount of ambient noised detected by the sensor 130or based on a currently-selected scene by a scene classifier of the userdevice 102.

In some examples, determining whether the broadcast sample 222 is adesired broadcast 212 includes asking the user. For example, the userdevice 102 can notify the user that a potentially acceptable broadcast212 has been found. The user device 102 can then ask the user whetherthe user would like to try that broadcast 212. Then, the use device 102can stimulate the user based on data received over the broadcast 212 fora period of time and then ask the user if her or she would like tocontinue to use the broadcast as a stimulation source. If the useragrees, then it can be determined that the broadcast sample 222 is adesired broadcast 212. In such an example, the sensor sample 242 andfingerprint 252 can be discarded, not collected, or unused. In furtherexamples, the sensor sample 242 and fingerprint 252 are used as aninitial, rough comparison and then, responsive to the initial comparisonpassing, the user is asked.

In some examples, the determining of a match is performed insubstantially real-time on the broadcast sample 222 and the sensorsample 242 as they are received. In some examples, processing isperformed to account for delay between the two samples 222, 242. Inother examples, the determining of a match is performed at a slightdelay, such as to permit processing on relatively longer samples 222,242.

In some examples, responsive to operation 260 indicating that there is amatch, the flow of the method 200 moves to operation 270. In someexamples, responsive to operation 260 indicating that there is not amatch, the flow of the method moves to operation 290. In some examples,responsive to operation 260 indicating that there is not a match, themethod 200 ends or loops back to the beginning. Returning to thebeginning can occur after a delay, a detected change in environment(e.g., a change in location as determined based on a location sensor ora change in a currently-classified scene), or in response to manualprompting by the user. In some instances, the broadcast 212 is one ofmultiple different potential broadcasts 212. In such an example,responsive to operation 260 indicating that there is not a match, themethod 200 returns to the beginning to try a different broadcast, suchas using one or more techniques described in more detail in the method300 of FIG. 3 , below.

Operation 270 includes selecting a broadcast 212 associated with thebroadcast sample 222. In an example, the operation 270 is performedresponsive to the broadcast sample 222 and the sensor sample 242matching in operation 260. For example, the selecting includes selectingthe broadcast 212 as the source of data on which stimulation provided bythe stimulator 140 is based. In some examples, operation 270 includesoperation 272. Operation 272 includes selecting the broadcast 212 as anaudio source, such as an audio source used to cause a person toexperience an audio percept. In addition or instead, operation 270includes selecting the broadcast 212 as a visual source used to causethe recipient of the user device to experience a visual percept.

In some examples, operation 270 includes changing a mode of operation ofthe user device 102. For example, the user device 102 first operates ina sensor mode where the user device 102 provides stimulation based onoutput from a sensor and then, as part of selecting the broadcast, theuser device 102 switches to operating in a broadcast mode where the userdevice 102 provides stimulation based on the broadcast 212. Anon-limiting example of operating a device in a microphone mode and anaccessory mode is described in U.S. Pat. No. 8,706,245, which waspreviously incorporated herein by reference. In an example, techniquesdescribed in U.S. Pat. No. 8,706,245 relating to the accessory mode canbe applied to operating based on the selected broadcast 212 andtechniques relating to the microphone mode can be applied to operatingbased on the sensor 130.

In an example, the determining (operation 260) and the selecting(operation 270) are performed by a secondary device 190, such asautomatically or manually by the secondary device 190. The operations260, 270 can be performed by any of a variety of devices or componentsof systems. In an example, the operation 260, 270 are performed by animplanted or external component of an auditory prosthesis (e.g., theuser device 102 implemented as an auditory prosthesis).

Operation 280 includes causing a person to experience a percept. In anexample, the percept is an audio percept, a visual percept, or bothaudio and visual percepts based on an audio signal or visual signal ofthe broadcast 212. In an example, the operation 280 includes generatingstimulation using the stimulator 140. In an example, the operation 280includes generating air-conducted vibrations, bone-conducted vibrations,or electrical stimulation configured to cause the recipient toexperience an auditory precept. In an example, operation 280 includesgenerating visual signals or electrical stimulation configured to causethe recipient to experience a visual percept.

Operation 282 includes to stream an audio signal. For example, asecondary device 190 streams the audio signal to a separate hearingdevice (e.g., the user device 102 configured as a hearing device), suchas an audio signal that causes the separate hearing device to stimulatethe person to experience the audio percept. In some examples, thesecondary device 190 does not stream the audio signal for a longduration. In some examples, the secondary device 190 initially selectsthe broadcast 212 and then transfers the broadcast (e.g., via a syncsignal) to the user device 102 such that the user device 102 directlyreceives the broadcast 212 from the broadcast system 150 rather thanthrough the secondary device 190 as an intermediary. In other examples,causing the person to experience the percept includes the secondarydevice 190 acting as an intermediary to provide an audio signal based onthe broadcast 212 to the user device 102 to cause the user device 102 tostimulate the user based thereon.

In some examples, the user device 102 has different processing pathwaysor settings for signals obtained from the sensors 120 and the receiver130. A non-limiting example of such an arrangement is described in U.S.Pat. No. 8,706,245, which was previously incorporated herein byreference.

Operation 290 includes to cease causing a person to experience a perceptbased on the broadcast 212. For instance, the user device 102 alreadycauses a user to experience a percept based on the broadcast 212. But,in some examples (e.g., the person walks away from an area where thebroadcast 212 is relevant but nonetheless the broadcast is beingreceived), it can be desirable to cease causing a person to experience apercept. As illustrated, the operation 290 is performed responsive tothe broadcast sample 222 and the sensor sample 242 not matching inoperation 260. In some examples, further criteria exist, such as thebroadcast sample 222 and the sensor sample 242 not matching more than athreshold number of times or not matching for longer than a thresholdamount of time.

In some examples, responsive to the broadcast sample 222 and the sensorsample 242 not matching, the user device 102 or the secondary device 190asks the user whether the user would like to continue to use theselected broadcast 212. Then, responsive to the user indicating that heor she does not want to continue to use the selected broadcast 212,operation 290 is performed.

In some examples, further responsive to the broadcast sample 222 and thesensor sample 242 not matching, broadcast 212 is excluded fromconsideration from future analyses as a non-matching broadcast (e.g., toavoid repeatedly trying to see if the broadcast 212 matches). In someexamples, the broadcast 212 is excluded from consideration after acertain number of failed attempts.

Computer Readable Medium

FIG. 3 illustrates a computer-readable medium 300 having instructions302 stored thereon that, when executed by one or more processors 110,cause the one or more processors 110 to perform a method 300 thatincludes one or more operations.

The computer-readable medium 300 is a component of a device of thesystem 100, such as the user device 102 or the secondary device 190.Depending on the implementation, the computer-readable medium 300 is acomponent of various devices, such as a visual prosthesis, auditoryprosthesis, or a consumer audio device. The computer-readable medium isa transitory or non-transitory computer-readable medium. In an example,the computer-readable medium 300 is a component of an implanted orexternal component of an auditory prosthesis selected from the groupconsisting of: a cochlear implant, an electroacoustic device, apercutaneous bone conduction device, a passive transcutaneous boneconduction device, an active transcutaneous bone conduction device, amiddle ear device, a totally-implantable auditory device, amostly-implantable auditory device, an auditory brainstem implantdevice, a hearing aid, a tooth-anchored hearing device, and a personalsound amplification product.

The instructions 302 are processor-executable program instructions that,when executed by the one or more processors 110 cause the one or moreprocessors 110 to perform actions or operations, such as the describedin relation to the methods herein. The instructions 302 can configurethe one or more processors 110 to perform operations.

Operation 310 includes to check for a set of candidate broadcasts 312.In an example, the operation 310 is performed automatically, such as ata predetermined time interval. In some examples, the check is not merelywhether the candidate broadcasts 312 exist, but whether each respectivebroadcast 314 meet threshold criteria. For instance, the thresholdcriteria can include the ability of the device performing the operationsto usefully obtain data from the respective broadcast 314 (e.g., therespective broadcast 314 is not passcode protected with an unknownpassword, encrypted, or in an incompatible format). The thresholdcriteria can further include the broadcast 314 including useful data.For instance, the user device 102 may be configured to cause therecipient to experience auditory precepts, and the threshold criteriacan be that the respective broadcast 314 conveys auditory data.

Then, for each respective broadcast 314 in the set of candidatebroadcasts 312, one or more of operations 320, 330, 340, 350, 360, 370,and 380 are performed.

Operation 320 includes to obtain a broadcast sample 322 from therespective broadcast 314. In an example, the operation 320 includes toobtain data from a component configured to receive the respectivebroadcast 314 as radio waves of between 2.4 GHz and 2.5 GHz that encodea signal. In an example, the operation 320 includes to obtain data froma receiver compatible with at least the BLUETOOTH 5.2 specification. Inan example, the operation 320 includes to store a portion of respectivebroadcast content provided over the respective broadcast 314. Operation320 can include one or more aspects of operation 220 as described above.

Operation 330 includes to obtain a sensor sample 332 from a sensor 120.In an example, the operation 330 includes to store a portion of sensordata provided by the sensor 120. Operation 330 can include one or moreaspects of operation 240 as described above.

Operation 340 includes to process the sensor sample 332. In someexamples, the user device 102 is a device configured to obtain auditoryinput and provide an output based thereon, such as a hearing aid,cochlear implant, other auditory prostheses, and certain consumer audiodevices (e.g., having passthrough audio features to bypass noiseisolation). Such devices 102 can perform processing on the audio such asnoise cancelation, beamforming, equalization, or other processing.Additional techniques include: gain adjustments (e.g., multichannel gaincontrol), noise reduction operations, or signal enhancement operations(e.g., speech enhancement, wind reduction), other operations, orcombinations thereof, in one or more of the channels. Noise reductioncan include processing operations that identify unwanted components of asignal (e.g., noise components), and then subsequently reduce thepresence of these unwanted components. Signal enhancement can refer toprocessing operations that identify the target signals (e.g., speech ormusic) and then subsequently increase the presence of these targetsignal components. Speech enhancement is a particular type of signalenhancement. While such techniques can be beneficial, where sensorsamples 332 are obtained from a same audio processing pathway, suchprocessing techniques can undesirably affect the sensor sample 332 forthe purposes of comparing the sensor sample 332 (or a fingerprintthereof) with data received over the broadcast 314. For instance, thesensor sample 332 is affected such that the sensor sample 332 isincorrectly classified as not matching the broadcast sample 322. Inother instances, certain processing of output from the sensor 120beneficially cleans up the sensor sample 332 to improve the ability ofthe sensor sample 332 to correctly match or not match the broadcastsample 322. For example, the equalization to correct for bias in theoutput of the sensor 120, wind noise cancelation, or other processingcan be beneficial. In some examples, the sensor sample 332 is processedin a same way as other sensor samples of the user device 102 (e.g.,during normal use where output from the sensors 130 is used to providestimulation), is processed in partially the same way (e.g., certainprocessing is performed and certain processing that would normally beperformed is not performed), is processed in a different way (certainprocessing not normally applied is applied), or is substantiallyunprocessed. In an example, operation 340 includes processing the sensorsample 332 by applying beamforming processing to the sensor sample 332or applying wind noise reduction to the sensor sample 332. For instance,the sensor sample 332 begins as raw output from the sensor 120 and isprocessed by applying one or both of beamform processing and wind noisereduction.

In an example, to process the sensor sample 332 includes to apply afirst sound processing technique to an audio signal obtained from thesensor 120 to form a first processed audio signal. For instance, thefirst sound processing technique can include wind noise cancelation or aprocessing technique that does not substantially negatively affect theability to compare the sensor sample 332 and the broadcast sample 322.Then, a second sound processing technique is applied to the firstprocessed audio signal to form a second processed audio signal. Thesecond sound processing technique is a sound processing technique thatis likely to negatively affect the ability to compare the sensor sample332 and the broadcast sample 322. Then the user device 102 stimulates arecipient of the sensory prosthesis using the second processed audiosignal, and the first processed audio signal is used as the sensorsample 332. For instance, to determine whether the broadcast sample 322and the sensor sample 332 match includes to compare the broadcast sample322 with the first processed audio signal.

Operation 350 includes to generate a broadcast sample fingerprint 352from the broadcast sample 322. Operation 360 includes to generate asensor sample fingerprint 362 from the sensor sample 332. Theseoperations 350, 360 can include one or more aspects as described abovein relation to operations 230 and 250, above.

Operation 370 includes to compare 370 the broadcast sample 322 and thesensor sample 332. In an example, comparing the broadcast sample 322 andthe sensor sample 332 includes determining whether the samples 322, 332match responsive to the broadcast sample fingerprint 352 and sensorsample fingerprint 362 having more than a threshold amount ofsimilarity. The operation 370 can include one or more aspects asdescribed above in relation to operation 260.

Responsive to the operation 370 indicating that the broadcast sample 322and the sensor sample 332 match, the flow of the method 300 moves tooperation 380. Responsive to the operation 370 indicating that thebroadcast sample 322 and the sensor sample 332 not matching, the flow ofthe method 300 moves to operation 382.

Operation 380 includes to select the respective broadcast 314 responsiveto the comparing (operation 370) indicating that the broadcast sample322 and the sensor sample 332 match. For instance, the respectivebroadcast 314 as an audio source used to cause a person to experience anaudio percept. The operation 380 can include one or more aspects asdescribed above in relation to operation 270.

Operation 382 includes to select a next broadcast 212 of the set ofcandidate broadcasts 312 as the respective broadcast 314 and theoperations 320, 330, 340, 350, 360 are performed for the new respectivebroadcast 314. If the respective broadcast was the last respectivebroadcast 314 of the set of candidate broadcasts 312 (e.g., there is nonext broadcast), then the flow of the method 300 can return to operation310.

After selecting the respective broadcast in operation 380, the flow ofthe method 300 moves to operation 390.

Operation 390 includes to cause a person to experience an audio perceptbased on the selected respective broadcast 314. In an example, theoperation 390 includes operation 392. Operation 392 includes to streaman audio signal to a hearing device to cause the hearing device tostimulate the person to experience the audio percept. The operation 390can include one or more aspects as described above in relation tooperation 280.

Processors

FIG. 4 illustrates one or more processors 110 configured to perform oneor more operations. The one or more processors 110 can becommunicatively coupled to memory having stored thereon instructionsthat so configure the one or more processors 110. For instance, thememory can include instructions thereon that, when executed by the oneor more processors 110, cause the one or more processors 110 to performthe one or more operations herein. In an example, the operations includeoperations 320, 330, 340, 350, 360, 370, 380, and 392, such as describedin FIG. 3 . In some examples, the one or more processors 110 areconfigured to perform fewer or additional operations, such as otheroperations described herein.

Example Computing System

FIG. 5 illustrates an example of a suitable computing system 500 withwhich one or more of the disclosed examples can be implemented.Computing systems, environments, or configurations that suitable for usewith examples described herein include, but are not limited to, personalcomputers, server computers, hand-held devices, laptop devices,multiprocessor systems, microprocessor-based systems, programmableconsumer electronics (e.g., smart phones), network PCs, minicomputers,mainframe computers, tablets, distributed computing environments thatinclude any of the above systems or devices, and the like. The computingsystem 500 can be a single virtual or physical device operating in anetworked environment over communication links to one or more remotedevices. In examples, the user device 102, the broadcast system 150, andthe secondary device 190, include one or more components or variationsof components of the computing system 500.

In its most basic configuration, computing system 500 includes one ormore processors 110 and memory 116, which are described above inrelation to FIG. 1 . In the illustrated example, the system 500 furtherincludes a network adapter 506, one or more input devices 508, and oneor more output devices 510. The system 500 can include other components,such as a system bus, component interfaces, a graphics system, a powersource (e.g., a battery), among other components.

The network adapter 506 is a component of the computing system 500 thatprovides network access. The network adapter 506 can provide wired orwireless network access and can support one or more of a variety ofcommunication technologies and protocols, such as ETHERNET, cellular,BLUETOOTH, near-field communication, and RF (Radiofrequency), amongothers. The network adapter 506 can include one or more antennas andassociated components configured for wireless communication according toone or more wireless communication technologies and protocols.

The one or more input devices 508 are devices over which the computingsystem 500 receives input from a user. The one or more input devices 508can include physically-actuatable user-interface elements (e.g.,buttons, switches, or dials), touch screens, keyboards, mice, pens, andvoice input devices, among others input devices.

The one or more output devices 510 are devices by which the computingsystem 500 can provide output to a user. The output devices 510 caninclude, displays, speakers, and printers, among other output devices.

Example Devices

As previously described, the technology disclosed herein can be appliedin any of a variety of circumstances and with a variety of differentdevices. For example, the user device 102 can take the form of a varietyof different consumer devices or medical devices. Example consumerdevices include headphones, earbuds, personal sound amplificationproducts, wireless earbuds, or other consumer devices. Example medicaldevices include auditory prostheses and visual prostheses. Exampleauditory prostheses include one or more prostheses selected from thegroup consisting of: a cochlear implant, an electroacoustic device, apercutaneous bone conduction device, a passive transcutaneous boneconduction device, an active transcutaneous bone conduction device, amiddle ear device, a totally-implantable auditory device, amostly-implantable auditory device, an auditory brainstem implantdevice, a hearing aid, and a tooth-anchored hearing device. Examplevisual prostheses include bionic eyes.

Specific example devices that can benefit from technology disclosedherein are described in more detail in FIGS. 6-8 , below. For example,the techniques described herein can be used to select broadcasts formedical devices, such as an implantable stimulation system as describedin FIG. 6 , a cochlear implant as described in FIG. 7 , or a retinalprosthesis as described in FIG. 8 .

Example Device—Implantable Stimulator System

FIG. 6 is a functional block diagram of an implantable stimulator system600 that can benefit from the technologies described herein. In anexample, the user device 102 corresponds to the implantable stimulatorsystem 600. The implantable stimulator system 600 includes a wearabledevice 610 acting as an external processor device and an implantabledevice 650 acting as an implanted stimulator device. In examples, theimplantable device 650 is an implantable stimulator device configured tobe implanted beneath a recipient's tissue (e.g., skin). In examples, theimplantable device 650 includes a biocompatible implantable housing 602.Here, the wearable device 610 is configured to transcutaneously couplewith the implantable device 650 via a wireless connection to provideadditional functionality to the implantable device 650.

In the illustrated example, the wearable device 610 includes one or moresensors 120, a processor 110, memory 116, a transceiver 618, and a powersource 648. The one or more sensors 120 can be units configured toproduce data based on sensed activities. In an example where thestimulation system 600 is an auditory prosthesis system, the one or moresensors 120 include sound input sensors, such as a microphone. Where thestimulation system 600 is a visual prosthesis system, the one or moresensors 120 can include one or more cameras or other visual sensors. Theprocessor 110 can be a component (e.g., a central processing unit)configured to control stimulation provided by the implantable device650. The stimulation can be controlled based on data from the sensor120, a stimulation schedule, or other data. Where the stimulation system600 is an auditory prosthesis, the processor 110 can be configured toconvert sound signals received from the sensor(s) 130 (e.g., acting as asound input unit) into signals 651. The transceiver 618 is configured tosend the signals 651 in the form of power signals, data signals,combinations thereof (e.g., by interleaving the signals), or othersignals. The transceiver 618 can also be configured to receive power ordata. Stimulation signals can be generated by the processor 110 andtransmitted, using the transceiver 618, to the implantable device 650for use in providing stimulation. In the illustrated example, thetransceiver 618 includes the receiver 130.

In the illustrated example, the implantable device 650 includes atransceiver 618, a power source 648, a coil 656, and a stimulator 120that includes an electronics module 610 and a stimulator assembly 124.The implantable device 650 further includes a hermetically sealed,biocompatible housing enclosing one or more of the components.

The electronics module 610 can include one or more other components toprovide sensory prosthesis functionality. In many examples, theelectronics module 610 includes one or more components for receiving asignal (e.g., from one or more of the sensors 120) and converting thesignal into the stimulation signal 615. The electronics module 610 canfurther be or include a stimulator unit (e.g., stimulator unit 122). Theelectronics module 610 can generate or control delivery of thestimulation signals 615 to the stimulator assembly 612. In examples, theelectronics module 610 includes one or more processors (e.g., centralprocessing units or microcontrollers) coupled to memory components(e.g., flash memory) storing instructions that when executed causeperformance of an operation. In examples, the electronics module 610generates and monitors parameters associated with generating anddelivering the stimulus (e.g., output voltage, output current, or lineimpedance). In examples, the electronics module 610 generates atelemetry signal (e.g., a data signal) that includes telemetry data. Theelectronics module 610 can send the telemetry signal to the wearabledevice 610 or store the telemetry signal in memory for later use orretrieval.

The stimulator assembly 612 can be a component configured to providestimulation to target tissue. In the illustrated example, the stimulatorassembly 612 is an electrode assembly that includes an array ofelectrode contacts disposed on a lead. The lead can be disposedproximate tissue to be stimulated. Where the system 600 is a cochlearimplant system, the stimulator assembly 612 is insertable into therecipient's cochlea. The stimulator assembly 612 can be configured todeliver stimulation signals 615 (e.g., electrical stimulation signals)generated by the electronics module 610 to the cochlea to cause therecipient to experience a hearing percept. In other examples, thestimulator assembly 612 is a vibratory actuator disposed inside oroutside of a housing of the implantable device 650 and configured togenerate vibrations. The vibratory actuator receives the stimulationsignals 615 and, based thereon, generates a mechanical output force inthe form of vibrations. The actuator can deliver the vibrations to theskull of the recipient in a manner that produces motion or vibration ofthe recipient's skull, thereby causing a hearing percept by activatingthe hair cells in the recipient's cochlea via cochlea fluid motion.

The transceivers 618 can be components configured to transcutaneouslyreceive and/or transmit a signal 651 (e.g., a power signal and/or a datasignal). The transceiver 618 can be a collection of one or morecomponents that form part of a transcutaneous energy or data transfersystem to transfer the signal 651 between the wearable device 610 andthe implantable device 650. Various types of signal transfer, such aselectromagnetic, capacitive, and inductive transfer, can be used tousably receive or transmit the signal 651. The transceiver 618 caninclude or be electrically connected to the coil 656.

The coils 656 can be components configured to receive or transmit asignal 651, typically via an inductive arrangement formed by multipleturns of wire. In examples, in addition to or instead of a coil, otherarrangements are used, such as an antenna or capacitive plates. Themagnets can be used to align respective coils 656 of the wearable device610 and the implantable device 650. For example, the coil 656 of theimplantable device 650 is disposed in relation to (e.g., in a coaxialrelationship) with an implantable magnet set to facilitate orienting thecoil 656 in relation to the coil 656 of the wearable device 610 via theforce of a magnetic connection. The coil 656 of the wearable device 610can be disposed in relation to (e.g., in a coaxial relationship) with amagnet set.

The power source 648 can be one or more components configured to provideoperational power to other components. The power source 648 can be orinclude one or more rechargeable batteries. Power for the batteries canbe received from a source and stored in the battery. The power can thenbe distributed to the other components of the implantable device 650 asneeded for operation.

Example Device—Cochlear Implant

FIG. 7 illustrates an example cochlear implant system 710 that canbenefit from use of the technologies disclosed herein. For example, thecochlear implant system 710 can be used to implement the user device102. The cochlear implant system 710 includes an implantable component744 typically having an internal receiver/transceiver unit 732, astimulator unit 720, and an elongate lead 718. The internalreceiver/transceiver unit 732 permits the cochlear implant system 710 toreceive signals from and/or transmit signals to an external device 750.The external device 750 can be a button sound processor worn on the headthat includes a receiver/transceiver coil 730 and sound processingcomponents. Alternatively, the external device 750 can be just atransmitter/transceiver coil in communication with a behind-the-eardevice that includes the sound processing components and microphone.

The implantable component 744 includes an internal coil 736, andpreferably, an implanted magnet fixed relative to the internal coil 736.The magnet can be embedded in a pliable silicone or other biocompatibleencapsulant, along with the internal coil 736. Signals sent generallycorrespond to external sound 713. The internal receiver/transceiver unit732 and the stimulator unit 720 are hermetically sealed within abiocompatible housing, sometimes collectively referred to as astimulator/receiver unit. Included magnets can facilitate theoperational alignment of an external coil 730 and the internal coil 736(e.g., via a magnetic connection), enabling the internal coil 736 toreceive power and stimulation data from the external coil 730. Theexternal coil 730 is contained within an external portion. The elongatelead 718 has a proximal end connected to the stimulator unit 720, and adistal end 746 implanted in a cochlea 740 of the recipient. The elongatelead 718 extends from stimulator unit 720 to the cochlea 740 through amastoid bone 719 of the recipient. The elongate lead 718 is used toprovide electrical stimulation to the cochlea 740 based on thestimulation data. The stimulation data can be created based on theexternal sound 713 using the sound processing components and based onsensory prosthesis settings.

In certain examples, the external coil 730 transmits electrical signals(e.g., power and stimulation data) to the internal coil 736 via a radiofrequency (RF) link. The internal coil 736 is typically a wire antennacoil having multiple turns of electrically insulated single-strand ormulti-strand platinum or gold wire. The electrical insulation of theinternal coil 736 can be provided by a flexible silicone molding.Various types of energy transfer, such as infrared (IR),electromagnetic, capacitive and inductive transfer, can be used totransfer the power and/or data from external device to cochlear implant.While the above description has described internal and external coilsbeing formed from insulated wire, in many cases, the internal and/orexternal coils can be implemented via electrically conductive traces.

Example Device—Retinal Prosthesis

FIG. 8 illustrates a retinal prosthesis system 801 that comprises anexternal device 810, a retinal prosthesis 800 and a mobile computingdevice 803. The retinal prosthesis system 801 can correspond to the userdevice 102. The retinal prosthesis 800 comprises a processing module 825and a retinal prosthesis sensor-stimulator 890 is positioned proximatethe retina 891 of a recipient. The external device 810 and theprocessing module 825 can both include transmission coils 856 alignedvia respective magnet sets. Signals 851 can be transmitted using thecoils 856.

In an example, sensory inputs (e.g., photons entering the eye) areabsorbed by a microelectronic array of the sensor-stimulator 890 that ishybridized to a glass piece 892 including, for example, an embeddedarray of microwires. The glass can have a curved surface that conformsto the inner radius of the retina. The sensor-stimulator 890 can includea microelectronic imaging device that can be made of thin siliconcontaining integrated circuitry that convert the incident photons to anelectronic charge.

The processing module 825 includes an image processor 823 that is insignal communication with the sensor-stimulator 890 via, for example, alead 888 which extends through surgical incision 889 formed in the eyewall. In other examples, processing module 825 is in wirelesscommunication with the sensor-stimulator 890. The image processor 823processes the input into the sensor-stimulator 890, and provides controlsignals back to the sensor-stimulator 890 so the device can provide anoutput to the optic nerve. That said, in an alternate example, theprocessing is executed by a component proximate to, or integrated with,the sensor-stimulator 890. The electric charge resulting from theconversion of the incident photons is converted to a proportional amountof electronic current which is input to a nearby retinal cell layer. Thecells fire and a signal is sent to the optic nerve, thus inducing asight perception.

The processing module 825 can be implanted in the recipient and functionby communicating with the external device 810, such as a behind-the-earunit, a pair of eyeglasses, etc. The external device 810 can include anexternal light/image capture device (e.g., located in/on abehind-the-ear device or a pair of glasses, etc.), while, as notedabove, in some examples, the sensor-stimulator 890 captureslight/images, which sensor-stimulator is implanted in the recipient.

Similar to the above examples, the retinal prosthesis system 801 may beused in spatial regions that have at least one controllable networkconnected device associated therewith (e.g., located therein). As such,the processing module 825 includes a performance monitoring engine 827that is configured to obtain data relating to a “sensory outcome” or“sensory performance” of the recipient of the retinal prosthesis 800 inthe spatial region. As used herein, a “sensory outcome” or “sensoryperformance” of the recipient of a sensory prosthesis, such as retinalprosthesis 800, is an estimate or measure of how effectively stimulationsignals delivered to the recipient represent sensor input captured fromthe ambient environment.

Data representing the performance of the retinal prosthesis 800 in thespatial region is provided to the mobile computing device 803 andanalyzed by a network connected device assessment engine 862 in view ofthe operational capabilities of the at least one controllable networkconnected device associated with the spatial region. For example, thenetwork connected device assessment engine 862 may determine one or moreeffects of the controllable network connected device on the sensoryoutcome of the recipient within the spatial region. The networkconnected device assessment engine 862 is configured to determine one ormore operational changes to the at least one controllable networkconnected device that are estimated to improve the sensory outcome ofthe recipient within the spatial region and, accordingly, initiate theone or more operational changes to the at least one controllable networkconnected device.

* * *

As should be appreciated, while particular uses of the technology havebeen illustrated and discussed above, the disclosed technology can beused with a variety of devices in accordance with many examples of thetechnology. The above discussion is not meant to suggest that thedisclosed technology is only suitable for implementation within systemsakin to that illustrated in the figures. In general, additionalconfigurations can be used to practice the processes and systems hereinand/or some aspects described can be excluded without departing from theprocesses and systems disclosed herein.

This disclosure described some aspects of the present technology withreference to the accompanying drawings, in which only some of thepossible aspects were shown. Other aspects can, however, be embodied inmany different forms and should not be construed as limited to theaspects set forth herein. Rather, these aspects were provided so thatthis disclosure was thorough and complete and fully conveyed the scopeof the possible aspects to those skilled in the art.

As should be appreciated, the various aspects (e.g., portions,components, etc.) described with respect to the figures herein are notintended to limit the systems and processes to the particular aspectsdescribed. Accordingly, additional configurations can be used topractice the methods and systems herein and/or some aspects describedcan be excluded without departing from the methods and systems disclosedherein.

Similarly, where steps of a process are disclosed, those steps aredescribed for purposes of illustrating the present methods and systemsand are not intended to limit the disclosure to a particular sequence ofsteps. For example, the steps can be performed in differing order, twoor more steps can be performed concurrently, additional steps can beperformed, and disclosed steps can be excluded without departing fromthe present disclosure. Further, the disclosed processes can berepeated.

Although specific aspects were described herein, the scope of thetechnology is not limited to those specific aspects. One skilled in theart will recognize other aspects or improvements that are within thescope of the present technology. Therefore, the specific structure,acts, or media are disclosed only as illustrative aspects. The scope ofthe technology is defined by the following claims and any equivalentstherein.

1. A method comprising: determining that a broadcast sample and a sensorsample match; and selecting a broadcast associated with the broadcastsample responsive to the broadcast sample and the sensor sample match.2. The method of claim 1, further comprising: obtaining a broadcastsample fingerprint from the broadcast sample; and obtaining a sensorsample fingerprint from the sensor sample, wherein determining that thebroadcast sample and the sensor sample match include: determining thatthe broadcast sample and the sensor sample match responsive todetermining that the broadcast sample fingerprint and the sensor samplefingerprint have a level of similarity within a threshold amount.
 3. Themethod of claim 1, wherein selecting the broadcast includes: selectingthe broadcast as an audio source used to cause a person to experience anaudio percept.
 4. The method of claim 1, further comprising: causing aperson to experience an audio percept based on an audio signal receivedover the broadcast.
 5. The method of claim 4, wherein the determiningand the selecting are performed by a computing device; and whereincausing the person to experience the audio percept includes: streaming,from the computing device to a separate hearing device, an audio signalthat causes the separate hearing device to stimulate the person toexperience the audio percept.
 6. The method of claim 1, furthercomprising: receiving the broadcast as radio waves having a frequencybetween 2.4 GHz and 2.5 GHz that encode a signal.
 7. The method of claim1, further comprising: receiving the broadcast as radio waves of between2.4 GHz and 2.5 GHz that encode a signal in compliance with theBLUETOOTH 5.2 specification.
 8. The method of claim 1, wherein thedetermining and the selecting occur automatically.
 9. The method ofclaim 1, further comprising: ceasing causing a person to experiencepercepts based on the broadcast responsive to the broadcast sample andthe sensor sample failing to match.
 10. The method of claim 1, whereinthe method further comprises obtaining the broadcast sample by storing aportion of broadcast content provided over the broadcast, and obtainingthe sensor sample by storing a portion of sensor data provided by asensor.
 11. A computer-readable medium having instructions storedthereon that, when executed by one or more processors, cause the one ormore processors to: check for a set of candidate broadcasts; and foreach respective broadcast of the set of candidate broadcasts: obtain abroadcast sample from the respective broadcast; obtain a sensor samplefrom a sensor; compare the broadcast sample and the sensor sample; andselect the respective broadcast responsive to the comparing indicatingthat the broadcast sample and the sensor sample match.
 12. Thecomputer-readable medium of claim 11, wherein the instructions, whenexecuted by the one or more processors cause the one or more processorsto, prior to comparing the broadcast sample and the sensor sample:process the sensor sample by: applying beamforming processing to thesensor sample or applying wind noise reduction to the sensor sample. 13.The computer-readable medium of claim 11, wherein the instructions, whenexecuted by the one or more processors cause the one or more processorsto: automatically perform the check for the set of candidate broadcastsat a predetermined interval.
 14. The computer-readable medium of claim11, wherein the instructions, when executed by the one or moreprocessors cause the one or more processors to: generate a broadcastsample fingerprint from the broadcast sample; and generate a sensorsample fingerprint from the sensor sample; and wherein to compare thebroadcast sample and the sensor sample includes to: determine that thebroadcast sample and the sensor sample match responsive to determiningthat the broadcast sample fingerprint and the sensor sample fingerprinthave a threshold amount of similarity.
 15. The computer-readable mediumof claim 11, wherein to select the respective broadcast includes to:select the respective broadcast as an audio source used to cause aperson to experience an audio percept.
 16. The computer-readable mediumof claim 15, wherein the instructions, when executed by the one or moreprocessors, cause the one or more processors to: cause the person toexperience an audio percept based on the respective broadcast, whereinto cause the person to experience the audio percept includes to: streaman audio signal to a hearing device to cause the hearing device tostimulate the person to experience the audio percept.
 17. Thecomputer-readable medium of claim 11, wherein to obtain the broadcastsample from the respective broadcast includes to: obtain data from acomponent configured to receive the respective broadcast as radio wavesof between 2.4 GHz and 2.5 GHz that encode a signal.
 18. Thecomputer-readable medium of claim 11, wherein to obtain the broadcastsample from the respective broadcast includes to: obtain data from areceiver compatible with at least the BLUETOOTH 5.2 specification. 19.The computer-readable medium of claim 11, wherein to obtain thebroadcast sample includes to store a portion of broadcast contentprovided over the respective broadcast; and wherein to obtain the sensorsample includes to store a portion of sensor data provided by thesensor.
 20. The computer-readable medium of claim 11, wherein theinstructions, when executed by the one or more processors, further causethe one or more processors to cause a person to experience an audiopercept based on an audio signal received over the respective broadcast.21. A system comprising: a sensory prosthesis; a sensor; a receiver; andone or more processors configured to: obtain a broadcast sample from thereceiver; obtain a sensor sample from the sensor; determine whether thebroadcast sample and the sensor sample match; and select, as a sourceused by the sensory prosthesis to cause a person to experience a sensorypercept, a broadcast associated with the broadcast sample responsive tothe broadcast sample and the sensor sample matching.
 22. The system ofclaim 21, wherein the sensor is a microphone; and wherein the sensoryprosthesis is an auditory prosthesis.
 23. The system of claim 21,wherein the sensor is a camera; and wherein the sensory prosthesis is avisual prosthesis.
 24. The system of claim 21, wherein the sensoryprosthesis comprises the sensor, the receiver; and the one or moreprocessors.
 25. The system of claim 21, further comprising: a computingdevice, wherein the computing device comprises the one or moreprocessors.
 26. The system of claim 25, wherein the computing devicefurther comprises the receiver; and wherein the one or more processorsare further configured to: stream, from the computing device to thesensory prosthesis, a signal based on data from the broadcast.
 27. Thesystem of claim 21, wherein the receiver is configured to receive thebroadcast as radio waves of between 2.4 GHz and 2.5 GHz that encode asignal.
 28. The system of claim 21, wherein the one or more processorsare further configured to: obtain a broadcast sample fingerprint fromthe broadcast sample; obtain a sensor audio fingerprint from the sensorsample; and determine that the broadcast sample and the sensor samplematch responsive to determining that the broadcast sample fingerprintand the sensor audio fingerprint have a level of similarity within athreshold amount.
 29. The system of claim 21, wherein the sensoryprosthesis is configured to: process the sensor sample, wherein toprocess the sensor sample includes to: apply a first sound processingtechnique to an audio signal obtained from the sensor to form a firstprocessed audio signal; apply a second sound processing technique to thefirst processed audio signal to form a second processed audio signal;and stimulate a recipient of the sensory prosthesis using the secondprocessed audio signal; and wherein to determine whether the broadcastsample and the sensor sample match includes to compare the broadcastsample with the first processed audio signal.
 30. The system of claim21, wherein the one or more processors are configured to obtain thebroadcast sample by storing a portion of broadcast content provided overthe broadcast.